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CFO 15466 US 



IMAGE DISTRIBUTION SYSTEM, AND IMAGE DISTRIBUTION 
METHOD AND PROGRAM THEREFOR 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to an image 
distribution system, a method and a program therefor, 
and more particularly to an image distribution system 
in a virtual system formed on a computer network, a 

10 method and a program therefor. 
Related Background Art 

In organizations such as business firms, 
associations or public bodies, it is customary that an 
office space is secured by possession or by renting as 

15 the work space for employed laborers (hereinafter 
called workers) to be engaged in office works, 
marketing works or technical works not requiring a 
large facility, and the workers arrive and gather at 
the office space by public transportation or by private 

20 cars by a work start time and execute group work within 
the predetermined work time. Such office space is 
equipped with telephones , copying machines , facsimiles , 
computers, computer networks etc. for improving the 
efficiency of various works. 

25 In the history of human beings, the habit of 

such group work is a relatively new phenomenon adopted 
for the efficient operation of the factories since the 
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industrial revolution. In recent years, the drawbacks 
resulting from such group work such as conjection in 
commuting and air pollution caused by the increase of 
the private cars have become apparent, while the group 
5 work is becoming not essential for the organizations 
and for their workers as a result of availability of 
the communication infrastructure such as internet and 
of various communication technologies. Therefore the 
attention is being directed to the concept of dispersed 

10 work in which the workers belonging to an organization 
executes their jobs at their homes or in arbitrary 
locations close thereto whereby the objective of the 
organization is attained as a whole. 

In general, for realizing the dispersed work in 

15 an organization such as a business firm, there is 

adopted a method of utilizing a room of the home of 
each worker as a working space (called home office) , 
connecting plural home offices in distant locations by 
a communication channel and utilizing home-use 

20 communication terminals such as a telephone set and a 
facsimile apparatus and a communication application 
system. In this manner there is secured the 
communicating means among the workers. The 
communication channel can be composed, for example, of 

25 a public telephone network, an ISDN network or an 
internet exclusive line, and the communication 
application system can be composed of an e-mail system. 
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a www (worldwide web) or a video conference system. 

The worker may also utilize a satellite office 
or a telecottage prepared by the organization or the 
local community to which the worker belongs or, in case 
5 of a sales job or a maintenance job, the seat of a 

private car or a public transportation vehicle, to be 
used for visiting the customer, as a temporary office 
(called mobile office). 

For the purpose of smoothly realizing such 
10 dispersed work, there have been proposed various 

system. Fig. 18 shows a client image in a multi-spot 
video conference provided by White Pine Software Inc., 
U. S. This system realizes, on personal computers, a 
real-time conference utilizing images among workers in 
15 distant locations. 

The mutual exchange of the images among the 
dispersed workers through such system allows to realize 
so-called interactive awareness system thereby avoiding 
isolated or secluded feeling of the workers and 
20 stimulating active communication among the workers. 

However, since the purpose of use is different 
in case the video conference system is utilized as a 
main communication tool and in case it is utilized as 
an interactive awareness system, the transmitted image 
25 results in various limitations in relation to the 

communication band and the convenience of use if the 
conventional system is applied for such purpose. More 
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specifically, the user is required to adjust the camera 
setting and the compression parameter according to the 
function to be utilized. 

5 SUMMARY OF THE INVENTION 

An object of the present invention is to 
resolve all the drawbacks mentioned above or at least 
one thereof. 

Another object of the present invention is, in 

10 order to resolve such drawbacks, to provide an image 
distribution system capable of constructing a virtual 
space system on a data network and mutually 
communicating an appropriate image according to the 
position of the user in the virtual space, and a method 

15 and a memory medium therefor. 

The above-mentioned objects can be attained, 
according to the present invention, by an image 
distribution system composed of terminal apparatuses 
respective provided at plural users and a server 

20 apparatus connected to such plural terminal apparatuses 
through a communication channel for constructing a 
virtual space for distributing an image among the 
terminal apparatuses, wherein each of the terminal 
apparatuses comprises image obtaining means for 

25 obtaining an image of the user, transmission means for 
transmitting the image, obtained by the image obtaining 
means, to the server apparatus, reception display means 
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for receiving and displaying the image transmitted from 
the server apparatus, designation means for designating 
the position of the user in the virtual space, and 
control means for controlling the image obtaining means 
5 according to the position of the user in the virtual 
space . 

Still another object of the present invention 
is to provide an image of higher image quality. 

The above-mentioned objects can be attained, 

10 according to the present invention, by an image 

distribution system composed of terminal apparatuses 
respective provided at plural users and a server 
apparatus connected to such plural terminal apparatuses 
through a communication channel for constructing a 

15 virtual space for distributing an image among the 
terminal apparatuses, wherein each of the terminal 
apparatuses comprises image obtaining means for 
obtaining an image of the user, transmission means for 
transmitting the image, obtained by the image obtaining 

20 means, to the server apparatus, reception display means 
for receiving and displaying the image transmitted from 
the server apparatus, and designation means for 
designating the position of the user in the virtual 
space, and the server apparatus comprises image process 

25 means for processing on the image transmitted from each 
of the terminal apparatuses , distribution means for 
distributing the image processed by the image process 
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means to the terminal apparatuses, and control means 
for controlling the image process means according to 
the position of the user receiving the image 
distribution in the virtual space . 
5 Still other objects of the present invention 

and the features thereof will become fully apparent 
from the following detailed description of the 
embodiments to be taken in connection with the 
accompanying drawings . 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic block diagram showing the 
configuration of a first embodiment; 

Fig. 2 is a view showing the hardware 
15 configuration of a host server apparatus 12; 

Fig. 3 is a view showing the software 
configuration of a host server apparatus 12; 

Fig. 4 is a view showing the configuration of 
devices in user terminal apparatuses 30 to 34; 
20 Fig. 5 is a view showing an example of 

arrangement of user terminal apparatuses 30 to 34; 

Fig. 6 is a view showing an example of an image 
called office view, displayed on a virtual office 
display apparatus 44; 
25 Fig. 7 is a view showing an example of a 

virtual office display image called conference room; 

Fig. 8 is a flow chart showing operations 
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relating to obtaining, displaying of image data and 
image control in a user terminal apparatus; 

Fig. 9 is a flow chart showing another example 
of the operations of the embodiment; 
5 Figs. 10A, 10B and IOC are views showing an 

example of image processing in the operations shown in 
Fig. 9; 

Fig. 11 is a flow chart showing a 3rd example 
of the operations of the embodiment; 
10 Fig. 12 is a plan view showing the camera 

arrangement in case of setting plural video cameras; 

Fig. 13 is a flow chart showing the operations 
in case two video cameras are set ; 

Fig. 14 is a flow chart showing the operations 
15 of a user terminal in case a process according to the 
user position is executed in a host server apparatus 
12; 

Fig. 15 is a flow chart showing the operations 
of the host server apparatus 12 corresponding to Fig. 
20 14; 

Fig. 16 is a flow chart showing the operations 
of a user terminal in another example in case the 
process according to the user position is executed in 
the host server apparatus 12; 
25 Fig. 17 is a flow chart showing the operations 

of the host server apparatus 12 corresponding to Fig. 
16; and 
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Fig. 18 is a view showing an example of a 
client image in a conventional multi-spot video 
conference system. 



5 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Now the present invention will be clarified in 
detail by preferred embodiments thereof, with reference 
to the accompanying drawings . 

In the following description, the "position in 
10 virtual space" means, in case of constructing a virtual 
office space for example in a network, a position in a 
virtual space such as a "large office room" or a 
"conference room" in which the user is present (namely 
utilized by the user) , and the function provided by the 
15 system (for example a function provided by the 
interactive awareness system or by the video 
conference) is determined according to such position. 

Fig. 1 is a schematic block diagram showing the 
configuration of a virtual office system constituting 
20 an embodiment of the present invention. 

In a main office 10 (for example the main 
office of an organization such as a business firm 
introducing the dispersed office system) , there is 
installed a host server apparatus 12 which is connected 
25 to a LAN (local area network) 14 in the main office 10. 

User terminal apparatuses 16, 18 for the workers in the 
main office 10 are realized by combining a computer 20, 
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a user terminal apparatus software 22, a telephone set 
24, a video camera 26 and other devices. The telephone 
24 can be connected to the host server apparatus 12 
through a PSTN channel (public telephone channel) or an 
5 ISDN channel, and the computer 20 is connected to the 
host server apparatus 12 through the LAN 14. 

User terminal apparatuses 30, 32, 34 for the 
workers in home offices are respectively realized by 
combining a computer 36, a user terminal apparatus 

10 software 38, a telephone set 40, a video camera 42, a 
virtual office display apparatus 44 and other devices. 
The telephone 40 can be connected to the host server 
apparatus 12 through a PSTN channel (public telephone 
channel) or an ISDN channel. The virtual office 

15 display apparatus 44 is connected to the computer 36. 
Fig. 2 shows an example of the hardware 
configuration of the host server apparatus 12. A basic 
platform (BP) 12a is composed of a PC host server 
apparatus. A signal processing unit (SPU) 12b is a 

20 signal processing board composed for example of a high- 
performance DSP (digital signal processor) . The SPU 
12b is utilized for signal processing for example for 
image and voice at the server. The BP 12a is connected 
through a LAN board to the LAN 14 in the main office 

25 10. Such configuration is already known and there may 
also be employed other configurations. 

Fig. 3 is a schematic block diagram showing the 
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software configuration of the host server apparatus 12. 
The software installed in the host server apparatus 12 
includes, for example, software programs developed with 
the C+ + language or the like and already existing 
5 software programs, and, as the OS (operating system), 
there can be employed, for example, the Windows NT 
(trade name of Microsoft Inc., U. S.). In Fig. 3, 
there are shown a server manager 50, an SPU access 
library 52, drivers 54, 56, a mail transmission unit 

10 58, a DDL (dynamic link library) 60, a dynamic web 

server 62, a database connector 64, Windows NT (trade 
name of Microsoft Inc., U. S.) 66, and a database 68. 

Fig. 4 shows an example of the configuration of 
the user terminal apparatus 30. The computer 36 is 

15 composed, as already known, of a main body 70, a mouse 
72, a keyboard 74, a display 76, a microphone 78, a 
speaker 80 and a modem (or a terminal adaptor) 82. The 
virtual office display apparatus 44 is composed of an 
image monitor apparatus such as a CRT display or a 

20 liquid crystal display and is connected to the main 
body 70 for displaying a virtual office image. The 
video camera 32 is also connected to the main body 70. 
The mouse 72, keyboard 74, display 76, microphone 78, 
speaker 80 and modem 82 are connected to corresponding 

25 input/output terminals of the main body 70. 

Fig. 5 shows an example of installation of the 
devices shown in Fig. 4, in a home office. In this 
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example, the virtual office display apparatus 44 is 
installed in a position different from that of the 
display 76 of the computer 36 to be used by the user 
for the works. Thus, when the user utilizes the main 
5 body 70 of the PC, the display area for the work is not 
hindered by the display for showing the office space 
thereby providing a comfortable environment, and, if 
the virtual office display apparatus 44 becomes larger, 
there can be realized an atmosphere closer to the 
10 reality. 

In the following there will be explained the 
function of the present embodiment . For the purpose of 
simplicity, it is assumed that the users of the home 
offices have already started to work by completing the 

15 connection to the host server apparatus 12 utilizing 
the respective user terminal apparatuses 30, 32, 34. 

Fig. 6 shows an image, called office view, 
displayed on the virtual office display apparatus 44. 
There are shown an image 90 showing the state of work 

20 of another user, a work state data display portion 92 
for displaying the work state data of such user and 
comments by text information, and private office space 
94 of such user. These three displays in combination 
constitute a virtual private office of the user. Fig. 

25 6 shows nine private offices on a same image, but the " 
number of such private offices may be selected larger 
or smaller. The space for displaying such private 
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offices is called a private office area 96. A space 
between the private offices is a virtual corridor. The 
user work state image 90, displayed as a part of the 
private office, is taken by the camera 42 provided in 
5 the user terminal apparatus 30 to 34 used by such user. 

At the right-hand side of the private office 
area 96, there is provided a common area 98, in which 
displayed are icons 100 = 108 respectively representing 
a conference room A, a conference room B, a laboratory, 

10 a refreshing room and a video news room. 

When the worker is working, the virtual office 
display apparatus 44 displays the office view as shown 
in Fig. 6. When the user is positioned in the office 
area in the virtual space, the system functions as an 

15 interactive awareness system so that the exchanged 

image can be of such content as to show the outlined 
state of the user. 

Fig. 7 shows an example of the virtual office 
display image called conference room. When the user 

20 clicks an conference room icon 100 on the office view, 
the display shifts to the conference room on the 
virtual space, as shown in Fig. 7. In a conference 
room display area 110 there are displayed images 112 of 
other participants and an image 114 schematically 

25 showing a table. An image 116 of the user is taken by 
the camera provided in the user terminal used by such 
user. A reference display area 118 displays references 
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to be used at the conference and registered in advance 
by the participants. 

Each participant executes the conference by 
looking at the image and the reference, utilizing 
5 unrepresented voice conference tool (also called voice 
chat tool) and character conference tool (also called 
text chat tool) . 

The voice conference tool includes a voice 
conference client software (voice telephone conference 

10 software for TCP/IP network based on the ITU-T standard 
H.323) provided in the user terminal apparatus and a 
voice conference server software (multi-spot voice 
telephone conference software for TCP/IP network based 
on the ITU-T standard H.323) provided in the host 

15 server apparatus, and, when these softwares are 

activated, the users can have a voice conference in 
multiple locations utilizing the microphone and the 
speaker provided in each user terminal apparatus . 

The text conference tool includes a text 

20 conference client software (text conference client 

software for TCP/IP network based on the IRC (internet 
relay chat) standard) provided in the user terminal 
apparatus and a text conference server software (multi- 
spot text conference server software for TCP/IP network 

25 based on the IRC conference) provided in the host 
server apparatus, and, when these softwares are 
activated, the users can have a text conference in 
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multiple locations utilizing the keyboard of each user 
terminal apparatus and a text chat window displayed 
simultaneously on the display separately from the 
conference room image (Fig. 7). 

The voice conference tool, the text conference 
tool and the multi-user game tool can be based on the 
known technologies , and will not be explained further 
since they are not related with the present embodiment. 

When the user is present in the conference 
room, it is important, for attaining smooth 
communication, to clearly transmit the face of the 
participant, thereby enabling to recognize the 
expression of each participant. 

Fig. 8 is a flow chart showing the operations 
relating to the obtaining and displaying image data and 
image control in the user terminal apparatus . In the 
following there will be explained the image 
distribution control operation of the virtual office 
system of the present embodiment, with reference to the 
flow chart shown in Fig. 8. 

The image taken by each video camera 42 is 
taken into the main body 70 of the computer by a video 
input device thereof, as image data of QCIF format (176 
x 144 pixels) (SI). Then the position of the user in 
the virtual space is judged (S2), and, in case of the 
office view (namely in case the user is present in the 
virtual large office space), there is set a first 
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compression parameter (S3), but, if otherwise, there is 
set a second compression parameter (S4). The 
compression parameter is assumed to be a quantization 
coefficient. More specifically, in case the user is 
5 present in the office view, namely in case the user 

utilizes the present system as an interactive awareness 
system, there is selected a large quantization 
coefficient (S3), but, in other cases, for example if 
the user is present in the conference room, there is 

10 selected a small quantization coefficient, and a face 

area is detected for example a method reported by Yokoo 
and Ogiwara, "Extraction of plural face areas from a 
natural image utilizing a hereditary algorithm" , Den- 
Gaku-Ron 117-C, 9, pp. 1245-1252( ( 1997) and a smaller 

15 quantization coefficient is assigned to the face area 
(S4). 

The image data are compression encoded by an 
image encoding method such as JPEG or H.263, utilizing 
the designated quantization coefficient (S5). For 

20 example, in case of JPEG compression, the 

transformation coefficients obtained by color space 
conversion and DCT are quantized by the designated 
quantization coefficient and then compression by the 
Huffman encoding. The compressed image data are 

25 promptly transferred through the network to the host 
server apparatus 12 (S6). 

The user terminal receives the images of other 
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users from the host server apparatus 12 (S7), expands 
the image data of such images (S8) and displays them on 
the display apparatus 44 (S9). 

These operations are repeated until the user 
5 instructs the end of the process (S10). 

In the present embodiment, the taken image is 
compressed with an automatically predetermined 
quantization coefficient, according to the position of 
the user in the virtual space. In case the user 

10 utilizes the interactive awareness function, the 

compression is executed with the large quantization 
coefficient to display other users as rough images* On 
the other hand, in case of the conference in the 
virtual conference room, the images of other users are 

15 displayed with a high definition. Therefore, in case 
of utilizing the interactive awareness function, the 
data amount of the images of other users are reduced so 
that the communication band can be utilized 
efficiently. Also high-definition images can be 

20 exchanged in case of utilizing the communicating 

function principally based on voice and text, such as 
the conference. It is also possible to show the face 
area with a higher definition in case of utilizing the 
active communicating function, utilizing the face 

25 recognition technology. 

Such control of the compression rate according 
to the function corresponding to the position in the 
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virtual space allows to transmit the optimum image 
while efficiently utilizing the communication band. 

In the example shown in Fig, 8, the image 
compression rate is switched according to the user 
5 position, but it is also possible to switch either to 
cut out a principal portion of the taken image or to 
reduce the size of the taken image according to the 
user position. Fig. 9 shows a flow chart of such 
variation . 

10 A field wider than the image to be actually 

transmitted is taken by the video camera 42, and the 
data of such taken image are fetched into the main body 
70 of the computer by the video input device thereof 
(Sll). In this operation, there is for example taken a 

15 full scene indicating the state of the user. 

Then the position of the user in the virtual 
space is judged (S12), and, in case the user is present 
in the office view (namely in case the user utilizes 
the present system as an interactive awareness system, 

20 the fetched image is linearly reduced to a size 

corresponding to the transfer rate of the communication 
channel (S13), but, in other cases, for example if the 
user is present in the conference room, an area 
principally containing the user face is cut out from 

25 the fetched image (S14). The cut-out area may be 

predetermined by the user, or the face area of the user 
may be automatically detected and cut out based on the 
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face recognition technology as explained in relation to 
Fig. 8. 

Figs. 10A to IOC show examples of image 
reduction (S13) and face cut-out (S14). Fig. 10A 
5 indicates a taken image, while Fig. 10B shows an image 
reduced to a desired size from the image shown in Fig. 
10A and Fig. IOC shows an image obtained by cutting out 
a face area of a desired size from that shown in Fig. 
10A. 

10 Such reduced image (S13) or cut-out image (S14) 

is compression encoded by an image encoding method such 
as JPEG or H.263 (S15), and the compressed image data 
are promptly transferred through the network to the 
host server apparatus 12 (S16). The user terminal 

15 receives the images of other users from the host server 
apparatus 12 (S17), expands the image data of such 
images (S18) and displays them on the display apparatus 
44 (S19). 

These operations are repeated until the user 
20 instructs the end of the process (S20). 

The convenience of use can be improved since it 
is automatically selected either to reduce the image or 
to cut out the predetermined portion so as to enable 
recognition of the user face, according to the user 
25 position in the virtual space. For example in case the 
user utilizes the interactive awareness function, the 
full- scene images of other users are received as rough 



images- On the other hand, in case of the conference 
in the virtual conference room, other users are 
displayed by images principally composed of the faces 
thereof. Therefore, in case of utilizing the awareness 
mode, it is possible to understand the status of other 
user, and, in case of utilizing the function 
principally based on communication such as the 
conference, satisfactory communication can be realized 
by high-definition images principally based on the 
faces . 

Such control of the transmitted image area 
according to the function corresponding to the position 
in the virtual space allows to realize transmission 
control of the optimum image according to the purpose 
of use. 

Fig, 11 is a flow chart showing a 3rd example 
of the operations of the present embodiment. The user 
terminal judges the position of the user in the virtual 
space (S21). In case the user is present in the office 
view (S21) namely in case the user utilizes the 
interactive awareness function, the zooming and the 
image taking direction of the camera 42 are so 
controlled as to take a full scene indicating the state 
of the user (S22) . In other cases (S21), the zooming 
and the image taking direction of the camera 42 are so 
controlled as to take a narrow area principally 
containing the user face (S23). The camera 42 can be 
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composed of a camera in which the panning, tilting and 
zooming can be controlled externally- The image taking 
area may be predetermined by the user, or automatically 
selected. Particularly in case of taking an image 
principally consisting of the user face, the image 
taking area may be determined by automatically 
detecting the face area of the user utilizing the face 
recognition technology as already explained in relation 
to Fig. 8. 

The image is taken with thus set image taking 
area and is fetched into the main body (S24). The 
fetched image is compression encoded by an image 
encoding method such as JPEG or H.263 (S25), and the 
compressed image data are promptly transferred through 
the network to the host server apparatus 12 (S26). The 
user terminal receives the images of other users from 
the host server apparatus 12 (S27), expands the image 
data of such images (S28) and displays them on the 
display apparatus 44 (S29). 

These operations are repeated until the user 
instructs the end of the process (S30). 

As explained in the foregoing, the image taking 
area is selected according to the user position in the 
virtual space, and the full- scene image is transmitted 
in case the user utilizes the interactive awareness 
function, but the image principally composed of the 
face is transmitted in other cases. Therefore it is 



possible to understand the state of other users in case 
of the awareness mode, and satisfactory communication 
is made possible by the high-definition image 
principally composed of the face in case of the 
function mainly for communication such as the 
conference. Also the process load is reduced since the 
image taking area can be selected solely by the camera 
control and does not required any particular image 
processing. 

Such control of the image taking device 
according to the function corresponding to the position 
in the virtual space allows to realize transmission 
control of the optimum image according to the purpose 
of use. 

As shown in a plan view in Fig. 12, the single 
video camera 42 may be replaced by a video camera 42a 
positioned in front of the user and adapted to take an 
image principally composed of the user face, and a 
video camera 42b taking the full scene where the user 
is present. There are also shown a video switching 
unit 120 for switching the outputs of the video camera 
42a, 42b, a user terminal 122 and a user 124. The 
video switching unit 120 is connected for example to a 
serial interface of the user terminal 122 and is 
controlled thereby. 

Fig. 13 is a flow chart showing the operations 
in case the two video cameras 42a, 42b are provided as 
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shown in Fig, 12, 

The user terminal 122 judges the position of 
the user in the virtual space (S31), and switches the 
video switching unit 120 according to the result of the 
5 judgment, thereby selecting the camera 42a or 42b to be 
used (S32). In case the user is present in the office 
view, namely in case the user utilizes the interactive 
awareness function, there is selected the camera 42b 
for taking the full scene indicating the state of the 

10 user. In this case, the full scene is taken from the 

back in order to reduce the psychological resistance of 
the user. On the other hand, in case the user is in 
the conference room, there is selected the camera 42a 
for taking the image of the narrow area principally 

15 composed of the user face. 

The image taken with the selected video camera 
42a or 42b fetched into the main body 70 (S33). The 
fetched image is compression encoded by an image 
encoding method such as JPEG or H.263 (S34), and the 

20 compressed image data are promptly transferred through 
the network to the host server apparatus 12 (S35). The 
user terminal receives the images of other users from 
the host server apparatus 12 (S36), expands the image 
data of such images (S37) and displays them on the 

25 display apparatus 44 (S38) ♦ 

These operations are repeated until the user 
instructs the end of the process (S39), 
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Such selection of the cameras 42a , 42b 
according to the user position in the virtual space 
allows to distribute the image showing the desired 
area. For example, the full scene of the user is 
distributed as a rough image in case the user utilizes 
the interactive awareness function, but the image 
principally showing the user face is displayed in case 
of the conference in the virtual conference room. 
Therefore, it is possible to understand the state of 
other users in case of the awareness mode, and the 
images principally composed of faces, suitable for 
communication are exchanged in case of the function 
mainly for communication such as the conference. 

As explained in the foregoing, the load of 
signal processing can be reduced since the image taking 
area is controlled solely by the camera selection and 
do not require any particular signal processing. Also 
the use of plural cameras increases the freedom in 
setting the image taking angle corresponding to the 
function to be used, and the convenience of use can be 
improved particularly in reducing the psychological 
burden of the user. 

In the following there will be explained an 
example of process according to the user position by 
the host server apparatus 12. Fig. 14 is a flow chart 
showing the operations of the user terminal, and Fig. 
15 is a flow chart showing the operation of the host 
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server apparatus 12. 

The image taken by each video camera 42 is 
taken into the main body 70 of the computer by a video 
input device thereof, as image data of QCIF format (176 
5 x 144 pixels) (S41). The fetched image data are 

compression encoded by an image encoding method such as 
JPEG or H.263 (S42), and are transferred through the 
network to the host server apparatus 12 (S43). 

The host server apparatus 12 receives the image 

10 transmitted from the client (S51), and judges the 
position of the user in the virtual space in the 
transmitting terminal (S52). In case the user of the 
transmitting terminal is present in the conference room 
(S52), the received image data are directly transmitted 

15 to the user (S56). In case the user of the 

transmitting terminal is present in the office view 
(S52), the received image data are partially expanded. 
For example, in case of the image data compressed with 
the JPEG method, the decoding is executed by the 

20 Huffman decoding and the inverse DOT to the DCT 

coefficients . Then the restored DCT coefficients are 
again quantized with a quantization coefficient larger 
than that at the compression encoding (S54), and are 
compression encoded again by entropy encoding such as 

25 Huffman encoding (S55). The host server apparatus 12 
distributes the image data thus subjected to partial 
decoding and recompression to the user terminal through 
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the network (S56). 

The user terminal receives the images of other 
users from the host server apparatus 12 (S44), expands 
the image data of such images (S45) and displays them 
5 on the display apparatus 44 (S46). 

The user terminal repeats the process of the 
foregoing steps S41 to S46 until the user instructs the 
end of the process (S47). The host server apparatus 12 
repeats the process of the foregoing steps S51 to S56 

10 until the user instructs the end of the process (S57). 

In the operations shown in Figs. 14 and 15, the 
host server apparatus 12 recompresses the image data in 
adaptive manner according to the user position in the 
virtual space. For example, in case the user utilizes 

15 the interactive awareness function, the host server 

apparatus 12 recompresses the image data with the large 
quantization coefficient, but, in case of the 
conference in the virtual conference room, the image 
data are transmitted without recompression. Therefore, 

20 in case of utilizing the interactive awareness 

function, the user receives the rough images of other 
users, but, in case of the virtual conference, receives 
the high-definition images of other users . In this 
manner, it is rendered possible to reduce the data 

25 amount of other users thereby effectively utilizing the 
communication band in case of the awareness mode, and 
to distribute the high-definition image in case of 
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utilizing the function principally for communication 
such as the conference* 

The control of the quantization coefficient by 
the server 12 allows to distribute optimum image data 
5 to each user* More specifically, it is possible to 

distribute the high-definition image data only to the 
user present in the conference room and to distribute 
the highly compressed image, including the images of 
other users present in the conference room, to the user 

10 present in the office view. 

The control of the quantization coefficient in 
the server according to the position in the virtual 
space allows to realize the optimum image distribution 
while achieving effective utilization of the 

15 communication band. 

In the example shown in Fig. 15, the image 
compression rate is changed by the host server 
apparatus 12 according to the user position of the 
transmitting terminal, but it is also possible that the 

20 server apparatus 12 reduces the image size in case of 
the office view but cuts out a desired partial image 
(for example a portion of the user face) in other 
cases. Fig. 16 is a flow chart showing the operations 
of the client in such variation, and Fig. 17 is a flow 

25 chart showing the operation of the host server 
apparatus 12 . 

The image taken by each video camera 42 is 
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taken into the main body 70 of the computer by a video 
input device thereof, as image data o£ QCIF format (176 
x 144 pixels) (S61) * The fetched image data are 
compression encoded by an image encoding method such as 
5 JPEG or H. 263 (S62) , and are transferred through the 
network to the host server apparatus 12 (S63). 

The host server apparatus 12 receives the image 
transmitted from the client (S71), and executes partial 
expansion (S72). For example, in case of the image 

10 data compressed by the JPEG method, the decoding is 

executed by Huffman decoding and inverse DCT to the DCT 
coefficients* Then the host server apparatus 12 judges 
the position of the user of the transmitting terminal 
in the virtual space (S73) - 

15 In case the user of the transmitting terminal 

is present in the office view (S73) r namely if the user 
utilizes the present system in the interactive 
awareness mode, the image size is reduced by deleting a 
part of the DCT coefficients according to the image 

20 size on the communication channel, in order to transmit 
the full scene showing the user state (S74). For 
example, the reduction is made to the QCIF format (176 
x 144 pixels), corresponding to the image shown in Fig. 
10B. 

25 On the other hand, in case the user is present 

in the conference room (S73), an area principally 
consisting of the user face is cut out from the 
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received image data (S75)» The cut-out area may be 
determined in advance, or may be cut out automatically 
utilizing the face recognition technology explained in 
the foregoing. The cut-out image data are output ted 
5 with the QCIF format, corresponding to the image shown 
in Fig, IOC. 

The size-reduced image (S74) or the cut-out 
image (S75) is quantized, then recompressed (Huffman 
encoding) (S76) and distributed to the user terminal 

10 through the network (S77). 

The user terminal receives the images of other 
users from the host server apparatus 12 (S64), expands 
the image data of such images (S65) and displays them 
on the display apparatus 44 (S66). 

15 The user terminal repeats the process of the 

foregoing steps S61 to S66 until the user instructs the 
end of the process (S67). The host server apparatus 12 
repeats the process of the foregoing steps S71 to S76 
until the user instructs the end of the process (S77). 

20 In the operations shown in Figs. 16 and 17, the 

host server apparatus 12 changes the image size or 
compression rate of the image data in adaptive manner 
according to the user position in the virtual space. 
For example, the host server apparatus 12 distributes 

25 the full scene image in case the user utilizes the 
interactive awareness function, but distributes the 
image principally showing the user face in case of the 
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conference in the virtual conference room. Therefore, 
the user receives the image of other users as a rough 
image in case of utilizing the interactive awareness 
function, and the high definition images of other users 
5 in case of the virtual conference. In this manner, it 
is rendered possible to understand the state of other 
users in case of the awareness mode, and the images 
principally composed of faces, enabling to understand 
the expression, in case of the function mainly for 

10 communication such as the conference. 

The cutting out of the image by the server 12 
allows to distribute optimum image data to each user. 
More specifically, it is possible to distribute the 
image data principally composed of the face only to the 

15 user present in the conference room and to distribute 
the full -scene image, including the images of other 
users present in the conference room, to the user 
present in the office view. 

The control by the server for the area or 

20 quality of the image to be distributed according to the 
position in the virtual space allows to realize the 
necessary image distribution with suitable range and 
quality, while achieving effective utilization of the 
communicat ion band . 

25 Also the control of the camera itself, such as 

the direction or zooming thereof, according to the 
position in the virtual space, is advantageous for 
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providing the image of higher quality. In distributing 
for example a magnified image, there is required 
electronic enlargement or cut-out if the image taking 
position or the zooming cannot be controlled, leading 
5 to a lower image quality with a reduced number of 

pixels. On the other hand, the control on the image 
taking position or the zooming allows to reduce the 
electronic image processing and to provide higher image 
quality, 

10 In the foregoing, the JPEG and H.263 encoding 

methods have been cited as the examples of the image 
compression encoding method r but such examples are not 
restrictive and there may be employed various image 
compression encoding methods. 

15 Also there have been explained the office view 

corresponding to a virtual large space as an example of 
the space utilizing the interactive awareness function 
and the virtual conference room as an example of the 
space realizing the active communication, but the 

20 present invention is not limited to such examples and 

can adapt to various functions. For example, there can 
be conceived a training room, a resting room or a 
library virtually constructed on the network, as 
functions for realizing the active communication. 

25 Also not only to the office space, the present 

invention is applicable to communication spaces of 
other purposes such as a communication tool for a hobby 
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group . 

It is furthermore possible to adopt an image 
distributing method in which the image working methods 
and the camera selecting methods explained in the 
5 foregoing are suitably combined. For example, it is 
possible to flexibly adapt to various needs of the 
users by allowing a manager to arbitrarily select the 
selection of such methods. 

In the foregoing there have been explained 

10 cases where the operation of the user terminal is 
realized by the software on the personal computer. 
However the present invention is not limited to such 
cases. It is for example possible to construct the 
display apparatus as a stand-alone exclusive terminal 

15 and to connect it independently to the network. In 

such case, various signal processings and communication 
processes of the user side are all executed in such 
terminal . 

The present invention also includes a case 
20 where program codes of a software realizing the 
functions of the aforementioned embodiments are 
supplied to a computer of an apparatus or a system 
connected to various devices so operating such devices 
as to realize the functions of the aforementioned 
25 embodiments and the computer (CPU or MPU) of such 

apparatus of system operate the aforementioned devices 
according to the supplied program. 
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In such case, the program codes themselves of 
the software realize the functions of the 
aforementioned embodiments, and the program codes 
themselves or the means for supplying the computer with 
5 the program codes , for example the memory medium 
storing the program codes, constitute the present 
invention. The memory medium storing the program codes 
can be, for example, a floppy disk, a hard disk, an 
optical disk, a magnetooptical disk, a CD-ROM, a 

10 magnetic tape, a non-volatile memory card or a ROM. 

The present invention naturally includes such 
program codes not only in a case where the functions of 
the aforementioned embodiments are realized by the 
execution of the supplied program codes by the computer 

15 but also in a case where the program codes realize the 
functions of the aforementioned embodiments in 
cooperation with the OS (operating system) functioning 
on the computer or other application softwares. 

Furthermore, the present invention includes a 

20 case where the supplied program codes are once stored 
in a memory provided in a function expansion board of 
the computer or in a function expansion unit connected 
thereto and a CPU or the like provided in such function 
expansion board or function expansion unit executes all 

25 the processes or a part thereof according to the 

instruction of the program codes thereby realizing the 
functions of the aforementioned embodiments. 
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As will be easily understood from the foregoing 
description , the present invention can realize optimum 
image distribution according to the state of use of the 
users, in a virtual office system or the like to be 
5 utilized in a dispersed work environment such as works 
at home or satellite offices. 

Also the present invention allows to distribute 
the necessary image information with appropriate area 
and image quality, while effectively utilizing the 
10 communication band, by controlling such area or image 
quality of the image to be distributed at the server 
side . 

Also the control of the camera itself, such as 
the direction or zooming thereof, according to the 
15 position in the virtual space, is advantageous for 
providing the image of higher quality. 

The present invention is not limited to the 
foregoing embodiments but is subject to various 
modifications within the scope and spirit of the 
20 appended claims. 



